Skip to content

[skip ci] Document how to quickly check if jit .dasc files transpile, how to test the jit in different architectures. #7768

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Dec 19, 2021

Conversation

TysonAndre
Copy link
Contributor

Figured this out from the makefile - this may be useful for new contributors (of features that require modifying the jit)

Aside: I'd also figured out how to use qemu to emulate arm64 in a VM, though there are multiple steps, and the emulated VM would make building or running tests significantly slower.
(https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/ seems easier)

@TysonAndre TysonAndre changed the title [skip ci] Document how to quickly check if zend_jit_arm64.dasc transpiles [skip ci] Document how to quickly check if jit .dasc files transpile, how to test the jit in different architectures. Dec 12, 2021
read-only, which is sometimes the cause of opcache bugs.
- `--repeat 2` is optional, but used in CI since some JIT bugs only show up after processing a
request multiple times, e.g. due to a bug in the engine or bad data in the
runtime cache of the interpreter.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The actual reason is that the first request compiles the trace and the second executes it.

request multiple times, e.g. due to a bug in the engine or bad data in the
runtime cache of the interpreter.
- `-j$(nproc)` runs as many workers to run tests as there are CPUs.
- `ext/opcache/` is the folder with the tests to run, in this case opcache
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might want to also pass Zend/

itself. If no folders are provided, all tests are run.

When investigating test failures such as segmentation faults, adding
`-m --show-mem` may be useful to test with [valgrind](https://valgrind.org/) to detect out of bounds memory accesses.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We usually use --enable-address-sanitizer builds instead.

```
export LDFLAGS=-L/usr/lib/i386-linux-gnu
export CFLAGS='-m32'
export CXXFLAGS='-m32'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line probably isn't needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's copied from azure/i386/job.yml - it'll prevent issues if anyone adapts these instructions to --enable-intl. ./ext/intl/breakiterator/breakiterator_class.cpp

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe our CFLAGS also applies to C++ code, but I can't be bothered to double check...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ext/intl/intl_convertcpp.lo: /path/to/php-src/ext/intl/intl_convertcpp.cpp
	$(LIBTOOL) --mode=compile $(CXX) -Iext/intl/ -I/path/to/php-src/ext/intl/ $(COMMON_FLAGS) $(CXXFLAGS_CLEAN) $(EXTRA_CXXFLAGS)  -DU_NO_DEFAULT_INCLUDE_UTF_HEADERS=1 -DU_HIDE_OBSOLETE_UTF_OLD_H=1 -Wno-write-strings -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 -std=c++11  -DUNISTR_FROM_CHAR_EXPLICIT=explicit -DUNISTR_FROM_STRING_EXPLICIT=explicit -c /path/to/php-src/ext/intl/intl_convertcpp.cpp -o ext/intl/intl_convertcpp.lo  -MMD -MF ext/intl/intl_convertcpp.dep -MT ext/intl/intl_convertcpp.lo

main/strlcpy.lo: /path/to/php-src/main/strlcpy.c
	$(LIBTOOL) --mode=compile $(CC) -Imain/ -I/path/to/php-src/main/ $(COMMON_FLAGS) $(CFLAGS_CLEAN) $(EXTRA_CFLAGS) -DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 -c /path/to/php-src/main/strlcpy.c -o main/strlcpy.lo  -MMD -MF main/strlcpy.dep -MT main/strlcpy.lo

If I add CFLAGS=-Wunused and then run ./config.nice, the only place -Wunused gets added is CFLAGS_CLEAN.

CFLAGS = $(CFLAGS_CLEAN) -prefer-non-pic -static
CFLAGS_CLEAN = -fno-common -Wformat-truncation -Wlogical-op -Wduplicated-cond -Wno-clobbered -Wall -Wextra -Wno-strict-aliasing -Wno-unused-parameter -Wno-sign-compare -Wunused -fvisibility=hidden -O0 -Wimplicit-fallthrough=1 -g -DZEND_SIGNALS $(PROF_FLAGS)
CPP = cc -E
CPPFLAGS = -D_GNU_SOURCE
CXX = g++
CXXFLAGS = -g -O0 -prefer-non-pic -static $(PROF_FLAGS)
CXXFLAGS_CLEAN = -g -O0

[AddressSanitizer](https://github.com/google/sanitizers/wiki/AddressSanitizer) is often useful.

Some of the time, adding `-m --show-mem` to the `TESTS` configuration is also useful to test with [valgrind](https://valgrind.org/) to detect out of bounds memory accesses.
This is less effective at detecting or explaining invalid memory accesses than AddressSanitizer, but does not require rebuilding php.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really less effective, just much slower. You can rebuild PHP in the time it takes to run a handful of tests under valgrind ;)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm#short-version

I mean that I thought AddressSanitizer would detect the computation of an invalid address, which might be distanced from the use (read/write) of the invalid address that caused misbehaviors.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, address sanitizer does not detect invalid address calculations. Though I believe ubsan does, at least to a limited degree.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh. Missed the * when reading that.

```
export LDFLAGS=-L/usr/lib/i386-linux-gnu
export CFLAGS='-m32'
export CXXFLAGS='-m32'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe our CFLAGS also applies to C++ code, but I can't be bothered to double check...

@TysonAndre TysonAndre merged commit 9a59417 into php:master Dec 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants